Deep multimodal learning for residential building energy prediction

Abstract The residential sector has become the second-largest energy consumer since 1987 in UK. Approximately 24 million existing dwellings England made up over 32% of overall consumption 2020. A robust understanding buildings’ performance is therefore critical guiding proper home retrofit measures to accelerate towards meeting UK’s climate targets. substantial number predictions at a city scale rely on available data, e.g., Energy Performance Certificates (EPCs) and GIS products, develop statistical machine learning models estimate consumption. However, issues with data are not negligible. This work adopted idea deep multimodal study potential for using Google Street View (GSV) images as an additional input building prediction. 20,031 GSV 5,933 buildings central Barnsley, UK, have been selected case study. All were pre-processed state-of-the-art object detection algorithm minimise noise caused by other elements that may appear nearby. Building specifications cannot be easily determined appearance extracted from EPC information text-based inputs model was designed jointly take texts inputs. These first propagated through convolutional neural network multi-layer perceptron, respectively, before being combined into connected final multi-input trained tested area predicted annual mean absolute difference 0.01kWh/m 2 per annum average compared what recorded EPC. between results also provide some hints bias certificates potentially contain.